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Abstract 

Motivated by EEG recordings of normal brain activity, we construct arbitrarily 
large McCulloch-Pitts neural networks that, without any external input, make 
every subset of their neurons fire in some iteration (and therefore in infinitely 
many iterations). 

1 Introduction 

Epilepsy is a group of neurologic conditions, the common and fundamental 
characteristic of which is recurrent, unprovoked epileptic seizures. These 
seizures are transient changes in attention or behavior, often accompanied by 
convulsions; they result from excessive, abnormal firing patterns of neurons 
that are located predominantly in the cerebral cortex (the convoluted outer 
layer of gray matter that covers each cerebral hemisphere). Such abnormal 
paroxysmal activity is usually intermittent and self-limited. [H p. 2]. The 
World Health Organization reports [26j that there are over 50 million epilepsy 
sufferers in the world today, 85% of whom live in developing countries. 

In attempts to study epilepsy, selected patients are monitored continu- 
ously for days at a time. During these periods, EEG (electroencephalogram) 
or ECoG (electrocorticogram) recordings are made. EEG recordings come 
from placing multiple electrodes on the scalp of the patient. ECoG record- 
ings produce far more accurate data, but they require invasive surgery to 
place a grid of electrodes directly on the cortex. 

Neurologists specialized in epilepsy are trained to read EEG/ECoG record- 
ings, so that mere visual inspection allows them to tell with a reasonable de- 
gree of accuracy when a seizure might have occured. There are a number of 
different types of seizures; two major categories are partial seizures (originat- 
ing in a small group of neurons, called a seizure focus, and spreading to other 
brain regions) and generalized seizures (showing simultaneous disruption of 
normal brain activity in both cerebral hemispheres from the onset) [25]. The 
classification [3J developed by the International League Against Epilepsy in 
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1981 divides each of these two categories into several subcategories, some of 
which are divided into subsubcategories or even further and these different 
types of seizures have different EEG manifestations |5l|20]. One frequent oc- 
currence is a transition from an irregular, disorderly EEG before the seizure 
(the pre-ictal state) to more organized sustained rhythm of spikes or sharp 
waves during the seizure (the ictal state) [21 Chapter 2]. (Some researchers 
refer to the pre-ictal EEG informally as 'chaotic' or 'random' in contrast 
with the rigorous definitions of these modifiers in mathematics, where — 
roughly speaking — 'chaotic' means 'highly dependent on initial conditions' 
and 'random' means 'unpredictable'.) 

In July 2009, in a seminar held at Concordia, Nithum Thain asked whether 
some initial configuration could cause Conway's Game of Life [7] to evolve 
in a way resembling a partial seizure, proceeding from an erratic flutter of 
apparently unpredictable patterns to sustained rhythmic changes that would 
begin in a small part of the grid and gradually spread, synchronized, over 
a larger area before subsiding to give way to the initial erratic mode. In 
the discussion that followed, a variation emerged: Could a McCuUoch-Pitts 
neural network behave like this? 

To define these networks, we need first the notion of a linear threshold 
function. This is a function / : R" — )■ {0, 1} such that, for some real numbers 
Wi,W2, . . . ,Wn (mnemonic for "weights") and 6 (mnemonic for "threshold"). 



f{Xi,X2, . . . , 




otherwise. 



The function is thought of as a neuron with zero-one signals Xi,X2, ■ ■ ■ ,Xn 
received at its synapses from the axons of other neurons; positive weights 
correspond to excitatory synapses and negative weights correspond to in- 
hibitory synapses; f{xi,X2, ■ ■ ■ , Xn) = 1 means that the neuron, given signals 
Xi,X2, ■ ■ ■ ,Xn at time t, will fire (send the signal 'one') along its axon after a 
synaptic delay at time t + 1. 

A McCulloch- Pitts neural network [llj, with nonnegative integer param- 
eters p and n such that p < n, is a collection of linear threshold functions 

/i:{0,ir^{0,l} {t = p+l,p + 2,...,n). 

Given any zero-one vector Sp+i, Sp+2, . . . , s„ (the initial state of the network) 
and any sequence ^i, ^2, ■ ■ ■ of p functions 

e.:N^{0,l} (r = l,2,...,p). 



2 



it computes a sequence Xp+i, Xp+2, . . . , x„ of n — p functions 
Xi:N^{0, 1} {i = p + l,p + 2,...,n.) 
This is done by setting Xj(0) = Sj and, for all nonnegative integers t, 

Xi{t+1) = fi{^i{t),...,^p{t),Xp+i{t),...,Xn{t)). 

We think of variable t as marking discrete time; each of the n — p neurons 
p + l,p + 2, . . . ,n may receive its signals from any of the n neurons; it receives 
a signal from neuron j if and only if its wj is nonzero. The bits C,r{t) with 
r = 1,2, .. .p and the bits Xi{t) with i = p + l,p + 2, . . . ,n tell us which 
neurons are firing at time t; firing or not firing of a neuron at time t + 1 
depends on the signals arriving to it at time t. Neurons 1,2,. . .p receive 
no signals; they have no axons synapsing upon them; McCulloch and Pitts 
call them 'the peripheral afferents' of the network. We will restrict ourselves 
to McCuUoch-Pitts neural networks with no such peripheral afferents; to 
put it differently, we will set p = 0. The entire network is then a function 
$ : {0, 1}" ^ {0, 1}" defined by 

<l>(x) = (/i(x),/2(x),...,/„(x)) 

and {xi(t),X2(t), . . .Xn(t)) = <l>*(si, S2 . . . , s„) for all nonnegative integers t. 

The McCulloch-Pitts concept of an artificial neuron has been generalized 
to allow firing at intensities on a continuous scale rather than in the all-or- 
none way: instead of being a linear threshold function, each : R" — )■ R is 
now defined by 

fi{Xi,X2, ...,Xn)= Vi{YTj=l WijXj) 

with some 'transfer function' yjj : R — )■ R; the McCulloch-Pitts original 
special case has v^i(s) = H{s — 9i), where H is the Heaviside step function 
defined by 

fl whenrf>0, 
[O when d < 0. 

All such mathematical abstractions of biological neurons are only crude ap- 
proximations of their actual behaviour. More credible models are spiking 
neurons, which were anticipated by Lapicque [12], [1] long before the mecha- 
nisms of the generation of neuronal action potentials were known; later on, a 
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different model was proposed by Hodgkin and Huxley [2] and subsequently 
elaborated by many other researchers [211 ED [HI [ISl IS] • 

We restrict ourselves to the McCuUoch-Pitts model (with no peripheral 
afferents) in all its simplicity: it played a seminal role in the development of 
artificial neural networks [221 123] and even today it is routinely referenced in 
medical hterature ([TUl [121 [2Z] are just three of the more recent citations). 
We do not pretend that our findings have any biological significance, but we 
hope that they may serve as a template for generalizations to more realistic 
models of the brain, such as networks of spiking neurons. 

2 The question 

Could a McCuUoch-Pitts neural network simulate a partial seizure? We 
have replaced this question by its easier variation: Could such a network 
simulate the pre-ictal state of a brain? To put it differently, are there ir- 
regular, disorderly, apparently unpredictable McCulloch-Pitts networks? An 
essential prerequisite of every such network $ : {0, 1}" — ?• {0, 1}" is that, 
starting from any state s in its domain {0, 1}", it eventually produce every 
state in this domain as an element of the trajectory s, . . . . This 

means that the period of^, defined as the smallest t such that $*+^(s) = s 
for some s in its domain, equals the size of the domain. This observation 
leads us to ask an even easier question: Are there McCulloch-Pitts networks 
$ : {0, 1}" {0, 1}" with period 2"? (The experimental study [18j of tra- 
jectories in randomly generated McCulloch-Pitts networks is to some extent 
related to this question; lengths of state cycles in Boolean networks, cellu- 
lar automata, and other finite dynamical systems are often interpreted as 
a measure of their computational power [13].) The result reported here is 
that the answer is affirmative for every positive integer n. (In addition, we 
have found a number of properties that threshold functions, and tuples of 
threshold functions, must possess in order to define such networks. These 
will be the subject of a subsequent paper.) 

3 The answer 

Given a positive integer n, we define a mapping : {0, l}" — ?■ {0, 1}" by 
$„(a;i, X2, . . . , Xn) = {yi,y2, ■ ■ ■ , Un) where, with m the largest subscript such 
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that (xi, X2, . . . , Xm) is an alternating vector, (0, 1, 0, 1, . . .) or (1, 0, 1, 0, . . .), 

(1) 

(Here, as usual, x with x G {0, 1} denotes 1 — x.) For instance. 



when 1 < z < m, 
when m < i < n. 



$4(0,0,0,0) = 


(1,0,0,0), 


$4(0,0,0,1) = 


(1, 


0, 


0, 


1 


$4(0,0,1,0) = 


(1,0,1,0), 


$4(0,0,1,1) = 


(1, 


0, 


1, 


1 


$4(0,1,0,0) = 


(1,1,1,0), 


$4(0,1,0,1) = 


(0, 


0, 


0, 





$4(0,1,1,0) = 


(0,0,1,0), 


$4(0,1,1,1) = 


(0, 


0, 


1, 


1 


$4(1,0,0,0) = 


(1,1,0,0), 


$4(1,0,0,1) = 


(1, 


1, 


0, 


1 


$4(1,0,1,0) = 


(1,1,1,1), 


$4(1,0,1,1) = 


(0, 


0, 


0, 


1 


$4(1,1,0,0) = 


(0,1,0,0), 


$4(1,1,0,1) = 


(0, 


1, 


0, 


1 


$4(1,1,1,0) = 


(0,1,1,0), 


$4(1,1,1,1) = 


(0, 


1, 


1, 


1 



Note that the definition of $„ implies that $„ is antisymmetric in the sense 
of 



and that 



$„(x) = $„(x) for all X in {0, 1}" 
$i(0) = l 



(2) 
(3) 



and that, when n > 2, 
$„(xi,a;2, . . .,Xn-i,0) 



(1, 1, ... , 1, 1) if {xi,X2, . . . , Xn-i, 0) is 

the alternating vector (. . . , 1, 0, 1, 0), (4) 
($„_i(a;i,X2, . . .,Xn-i),0) otherwise. 

Theorem 1. The period of is 2". 

Proof. Straightforward induction on n, using properties ([2]), ([3]), (jll), proves 
a finer statement: The 2" vectors $^(0, 0, . . . , 0) with t = 0, 1, . . . , 2" - 1 are 
pairwise distinct and $^(0, 0, . . . , 0) with t = 2" — 1 is the alternating vector 
(...,0,1,0,1). □ 

Theorem 2. For every positive integer n there are linear threshold functions 
/„, :{0,ir^{0,l} (z = l,2,...,n) 

such that 

^n{x) = ifn,l{x), fn,2{x), fn,n{x)) for all X lU {0, 1}". (5) 
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Proof. For every positive integer n and for alH = 1, 2, . . . , n, we will construct 
weights Wn,i, j (j = 1, 2, . . . , n) and threshold values 9n,i- Then we will define 

and prove that (jSD is satisfied. 

Our construction of WnA, j and 9n,i is recursive. To begin, we set 

= -I7 ^1,1 = 0; 
for all integers n greater than 1, we set 

1 if j ^ n mod 2, 

'Wn,n,j — { ~^ if j = n mod 2 and j < n, 

n — 2 if j = n, 
V= [n/2j, 

{W„_l,„_l,j if j < 72 - 2, 

-1 if j = n, 

9n,n—l ^n— 1,71—1) 

and, when i = 1, 2, . . . , n — 2, 

w„_i,i, J + w„-2,i, i if i < ^ - 2, 

Wn,i, j = { Wn-l,i, j if j = n- 1, 

-1 if j = n, 

Since the sequence $i, $2, $3, . . . is completely determined by its properties 
([2]), ([3]), (jl]), proving that ([5]) is satisfied reduces to proving that 



(i) fn,iix) = fn,ii^) for alH = 1, . . . and all x in {0, 1}" 
observing that /i,i(0) = 1, and proving that 

(ii) if X in {0, 1}" is the alternating vector (. . . , 1, 0, 1, 0), 
then fn,i{x) = 1 for alH = 1, . . . n. 
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(iii) if (xi, . . . , 0) in {0, 1}" is not the alternating vector (...,1,0, 1, 0), 

then fn,n{^l, • • • , Xn-l, 0) = 0, 

(iv) if (xi, . . . , Xn-i, 0) in {0, 1}" is not the alternating vector (...,1,0, 1, 0), 
then /n,i(xi, . . . , 0) = fn-i,ii^i, • • • , Xn-i) for alH = 1, . . . n - 1. 

Straightforward, if a little tedious, induction on n shows that 

Wn,i, j = On,i for alH = 1, . . . n, (6) 

j^n mod 2 

u!n,i, j = dn,i - 1 for alH = 1, . . . n. (7) 

j=n mod 2 

Summing up each pair of these equations, we conclude that 
Z]"=i Wn,i, j = '20n,i - 1 for alH = 1, . . . n, 

which is easily seen to imply (i); equations ([6]) alone imply directly (ii); 
proposition (iii) follows from the definitions of Wn,n,j and 6'„^„. 

In proving (iv), we will treat i = n — 1 separately from i < n — 2. 

To prove that fn,n-i{xi, Xa, • • • , 0) = /„_i,„_i(xi, Xa . . . , x„_i) for all 
(xi, . . . , Xn-i) in {0, 1}""^ other than the alternating vector (. . . , 0, 1, 0, 1), 
recall that 



j—l '^n,n—l,jXj j— ] Wn—l^n—l,jXj -\- Xn—l and 6n,n—l f^n — l,n— 1; 



It follows that /„,„_! (xi, . . . , Xn-i, 0) ^ fn-i,n-i{xi, . . . , Xn-l) if and only if 



X, 



„_i = 1 and XlLi tt^n-i^n-i, jXj + 1 = 6n-i,n~i- Siuce iy„_l,„-l,n-l = n - 3 



and On-i-n-i = [{n - 1)/2J, this means X]"=i Wn_i,n-i, = - \{n- 3)/2]; 
since 



■u;„_i,n-i, j 



1 if j ^ n — 1 mod 2, 

— 1 if j = n — 1 mod 2 and j < n — 1, 



this means further that (xi, . . . , x„_i) is the alternating vector (. . . , 0, 1, 0, 1). 

To prove that we have /„,i(xi, . . . , x„_i, 0) = /„_i,i(xi, . . . , x„_i) for all 
i = 1, . . . ,n — 2 and for all (xi, . . . , x„_i) in {0, 1}"^^ other than the alter- 
nating vector (. . . , 0, 1, 0, 1), we shall use induction on n. In the induction 
step, we distinguish between two cases. 
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Case 1: (xi, . . . is the alternating vector {... ,1,0,1,0). 

In this case, we do not use the induction hypothesis. Equations ([7]) show 
that 

I]"=l W^n,i, jXj + Wn,i, n = On,i - 1 for alH = 1 , . . . Ti; 
since Wn,i, „ = — 1 for alH = 1, . . . n — 1, it follows that 

Z)j=i^ri,i, jXj = 6n,i for alH = 1, . . . n - 1, 

and so fn,i{xi, . . . , Xn-i, 0) = 1 for alH = 1, . . . , n — 1. By (ii) with n — 1 in 
place of n, we have . . . , x„_i) = 1 for alH = 1, . . . , n — 1. 

Case 2: (xi, . . . is not the alternating vector {... ,1,0,1,0). 

In this case, consider an arbitrary {xi, . . . , x„_i, 0) in {0, 1}" other than the 
alternating vector (...,1,0, 1, 0). The induction hypothesis guarantees (alone 
if Xn-i = and combined with (i) if x„-i = 1) that fn-i,i{xi, . . . ,Xn-i) = 
fn-2,i{xi, Xn-2) for alH = 1, . . . n - 2. 

If z < n - 2 and fn-iA^i, • • • , Xn-i) = 1, then /„-2,i(xi, . . . , Xn-2) = 1, 
and so 

En— 1 _l_ 2 

j=l — Z^j=i Wn-l,i,jXj + Z^j=i Wn-2,i,jXj 

> dn-l,i + Qn~2,i > On,i, 

which implies /„,i(xi, . . . , 0) = 1. 

If 2 < n - 2 and fn-i,i{xi, Xn-i) = 0, then /„_2,i(a;i, . . . , Xn-2) = 0, 
and so 



En— i Y^^""-!- I v~^n— 2 

j=l '^n,i,jXj — Z^j=i UJn-l^ijXj + Z^j=i Wn-2,i, jXj 

< (6'n-l,i — 1) + {Gn-2,i — 1) < ^n,i, 

which implies /„i(xi, . . . , x„_i, 0) = 0. □ 

Implicit in our proof of Theorem [1] is a simple way of transforming each 
trajectory 

(0,0,...,0)h>$„(0,0,...,0) H^... $^-1(0,0,..., 0) (8) 
with = 2" into the trajectory 

(0,0,...,0)^<l>„+i(0,0,...,0) ^...<l>^^r'(0,0,...,0) : (9) 
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First append as the last bit to each point of the trajectory ([H]) and let T 
denote the resulting sequence of 2" vectors (xi, . . . , x„, 0) in {0, l}""*"^; then 
flip every bit (0 <(-)■ 1) of every vector in T and let T denote the resulting 
sequence of 2" vectors (xi, . . . , Xn, 1) in {0, l}""*"^; the trajectory ([9]) is the 
concatenation TT. For instance, if = 3, then (|8]) is 

(0, 0, 0) ^ (1, 0, 0) ^ (1, 1, 0) ^ (0, 1, 0) ^ 
(1,1,1)^(0,1,1)^(0,0,1)^(1,0,1), 



(0, 0, 0, 0) ^ (1, 0, 0, 0) ^ (1, 1, 0, 0) h-> (0, 1, 0, 0) ^^ 
(1, 1, 1, 0) ^ (0, 1, 1, 0) ^ (0, 0, 1, 0) (1, 0, 1, 0), 



(1, 1, 1, 1) ^ (0, 1, 1, 1) ^ (0, 0, 1, 1) ^ (1, 0, 1, 1) ^ 
(0, 0, 0, 1) ^ (1, 0, 0, 1) ^ (1, 1, 0, 1) ^ (0, 1, 0, 1), 



(0, 0, 0, 0) ^ (1, 0, 0, 0) ^ (1, 1, 0, 0) ^ (0, 1, 0, 0) h-> 
(1, 1, 1, 0) ^ (0, 1, 1, 0) ^ (0, 0, 1, 0) ^ (1, 0, 1, 0) H-> 
(1, 1, 1, 1) ^ (0, 1, 1, 1) ^ (0, 0, 1, 1) ^ (1, 0, 1, 1) ^ 
(0, 0, 0, 1) ^ (1, 0, 0, 1) ^ (1, 1, 0, 1) ^ (0, 1, 0, 1). 

It may be interesting to note that our $„ can be specified in yet another 
way. Every one-to-one mapping r : {0, 1}" — )■ {0, 1, ... 2" — 1} generates a 
mapping $ : {0, 1}" {0, 1}" through the formula 

$(x) = r"^(r(x) + 1 mod 2"). 

We have, for s = r"^(0) and for allt = 0, 1, . . . 2" — 1, 

X = $*(s) <^t = r{x) 

(this can be checked by straightforward induction on t); it follows that $ has 
period 2". Our is generated by the mapping r„ : {0, 1}"" — >■ {0, 1, . . . 2" — 1} 
defined by r„(a;i, X2, ■ ■ ■ x„) = X]"=i Cj2^"^ with 




\xj — when 1 < j < n, 

Xn when j = n. 
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T is 



T is 



and (ED is 



For instance, 



r4(0, 0, 0, 0) = 0, r4(0, 0, 0, 1) = 12, n{0, 0, 1, 0) = 6, r^iO, 0, 1, 1) = 10, 

r4(0, 1, 0, 0) = 3, r4(0, 1, 0, 1) = 15, r4(0, 1, 1, 0) = 5, r4(0, 1, 1, 1) = 9, 

r4(l, 0, 0, 0) = 1, r4(l, 0, 0, 1) = 13, r4(l, 0, 1, 0) = 7, r4(l, 0, 1, 1) = 11, 

r4(l, 1, 0, 0) = 2, r4(l, 1, 0, 1) = 14, r4(l, 1, 1, 0) = 4, r4(l, 1, 1, 1) = 8 



This mapping r„ is one-to-one for every n: from the integer r„(xi, X2, . . . 
we can recover its binary encoding (ci, C2, . . . c„), from which we can recover 
first Xn, then and so on until Xi. 

To see that rn($„(x)) = r„(x) + l mod 2", observe that (i) if x is the alter- 
nating vector (. . . , 0, 1, 0, 1), then r„(a;) = 2" — 1 and (ii) for all other vectors 
(xi, X2, . . . , Xn) in {0, 1}", the largest subscript m such that (xi, X2, ■ ■ ■ , Xm) 
is an alternating vector equals the smallest subscript m such that = 0, in 
which case r„(xi, X2, ■ ■ ■ Xn) + 1 = X]?=i with 



and, with (yi, . . . Vn) defined by ([I]), we have r„(?/i, . . . yn) = YTj=i dj'^^ ^• 

4 How many n-neuron McCulloch-Pitts 
networks with period are there? 

When n = 2, there are just two such networks, 

{xi,X2) ^ {H{-X2),H{xi- I)) and {xi, X2) ^ {H {x2 - I) , H {-xi)) 

with H the Heaviside step function defined in Section [T] The first of these 
two networks is $2; the second is produced from $2 by switching the two 
coordinates. 

Next, let us consider n = 3. By permuting the three coordinates, $3 




when 1 < j < m, 

1 when j = m, 

Cj when m < j < n 
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produces six distinct McCuUoch-Pitts networks: 



000 100 110 -> 010 111 oil 001 101 000, 
000 ^ 010 -> 110 -> 100 -> 111 ^ 101 ^ 001 ^ oil ^ 000, 
000 001 ^ 101 ^ 100 ^ 111 ^ 110 ^ 010 ^ oil ^ 000, 
000 ^ 001 ^ oil ^ 010 ^ 111 ^ 110 ^ 100 ^ 101 ^ 000, 
000 ^ 010 ^ oil ^ 001 ^ 111 ^ 101 ^ 100 ^ 110 ^ 000, 
000 100 101 ^ 001 111 ^ oil ^ 010 110 000. 

(Here, we write for (xi, a;2, Xs) and we appeal to the fact that an 

n-neuron McCulloch-Pitts network with period 2" is fully specified by its 
trajectory.) By fiipping bits (0 -H- 1), these six McCulloch-Pitts networks 
produce an additional eighteen McCulloch-Pitts networks: flipping the flrst 
bit (and rotating the resulting trajectory to make 000 its starting point), we 
get 

000 ^ 010 ^ 110 ^ Oil ^ 111 ^ 101 ^ 001 ^ 100 ^ 000, 
000 ^ Oil ^ 001 ^ 101 ^ 111 ^ 100 ^ 110 ^ 010 ^ 000, 
000 ^ Oil ^ 010 110 ^ 111 ^ 100 101 ^ 001 ^ 000, 
000 ^ 001 ^ 100 101 ^ 111 ^ 110 oil ^ 010 ^ 000, 
000 010 ^ 100 ^ 110 ^ 111 ^ 101 ^ oil 001 ^ 000, 
000 001 ^ 101 oil ^ 111 ^ 110 010 100 ^ 000; 

flipping the second bit (and rotating the resulting trajectory to make 000 its 
starting point), we get 

000 101 001 oil 111 010 110 100 000, 

000 ^ 100 110 ^ 101 ^ 111 -> oil ^ 001 010 ^ 000, 
000 -> 001 -> 010 ^ oil 111 ^ 110 ^ 101 -> 100 ^ 000, 
000 ^ 101 ^ 100 ^ 110 ^ 111 ^ 010 ^ oil ^ 001 ^ 000, 
000 ^ 001 oil ^ 101 ^ 111 ^ 110 -> 100 ^ 010 ^ 000, 
000 ^ 100 ^ 010 ^ 110 ^ 111 ^ oil ^ 101 ^ 001 ^ 000; 

flipping the third bit (and rotating the resulting trajectory to make 000 its 
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starting point), we get 

000 100 001 101 111 oil 110 010 ^ 000, 
000 ^ 010 ^ 001 ^ oil ^ 111 ^ 101 ^ 110 ^ 100 ^ 000, 
000 ^ 100 ^ 101 ^ 110 ^ 111 ^ on ^ 010 ^ 001 ^ 000, 
000 010 oil ^ 110 111 101 100 001 000, 
000 110 100 101 111 001 ^ oil 010 ^ 000, 
000 ^ 110 ^ 010 ^ oil ^ 111 ^ 001 ^ 101 ^ 100 ^ 000. 

Since $3 is antisymmetric, flipping two or three bits produces no additional 
McCuUoch-Pitts networks. To summarize, $3 produces an isomorphism class 
of 24 networks, where 'isomorphism' means any composition of permuting 
subscripts and flipping bits. The McCuUoch-Pitts network defined by 

{xi,X2,X3) {H{-Xi+X2-Xs),H{-Xi-X2-X3 + l),H{-Xi+X2+X3-l)) 

does not belong to this class: its trajectory is 

000 ^ 110 ^ 100 ^ 010 ^ 111 ^ 001 ^ on ^ 101 ^ 000. 

By permuting subscripts and flipping bits, this new network produces a new 
isomorphism class of 24 networks. Computer search shows that there are 
no 3-neuron McCuUoch-Pitts networks with period 8 other than these 48 
networks in these 2 isomorphism classes. 

Additional computer search shows that there are precisely 9984 distinct 
4-neuron McCulloch-Pitts networks with period 16 and that these networks 
come in 56 distinct isomorphism classes. It seems that the number of iso- 
morphism classes of n-neuron McCulloch-Pitts networks with period 2" grows 
rapidly with n. 

5 Pseudorandom number generators 

Our has period 2", as long as could possibly be, but it is not quite 
irregular, disorderly, and apparently unpredictable. To point out two of its 
blatant blemishes, let denote the i-th bit of a vector y in {0, 1}". 
Our recursive description of the trajectory 

(0,0,...,0)^$„(0,0,...,0)^...$^-^(0,0,...,0) 
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with N = 2^ implies first that 

(0,0,...,0)^<l>„(0,0,...,0), <^^-\o,o,...,o)y^<i>^-\o,o,...,o) 

and then that 

xi = $„(a;)i $„(x)i ^ (10) 

Our recursive description of the trajectory also implies that for all i = 
1,2, ... ,n, the trajectory splits up into segments of length 2*^^ so that the 
i-th bit of $^(0, 0, . . . , 0) is constant in each segment; it follows that 

foralH = 2,3, ...,n, ^ <l>„(a;), ^ = (11) 

Properties (|T0|) . f fTTj) make far from irregular, disorderly, and apparently 
unpredictable: in a random permutation $ of {0, 1}"", we would expect Xi = 
$(x)i = ^^{x)i about 25% of the time and, for each i = 2, 3, . . . , n, we would 
expect Xi 7^ ^{x)i ^ $^(a;)j about 25% of the time. 

By definition, no computable function g : X ^ X (where X is a finite set 
of numbers) can generate random numbers in the sequence s, g{s), g'^{s), . . .: 
if g{x) can be computed, then it is not unpredictable. (For an exposition 
of the concept of randomness, we recommend [B].) Functions g : X X 
that seem to generate random numbers are called pseudorandom number 
generators. Since each vector in {0, 1}" is a binary encoding of an n-hit 
nonnegative integer, every mapping $ : {0, 1}" — )■ {0, 1}" induces a mapping 
$* : {0, 1, . . . , 2" — 1} — > {0, 1, . . . , 2" — 1}, and so every irregular, disorderly, 
apparently unpredictable McCuUoch-Pitts network $ induces a pseudoran- 
dom number generator $*. Let us write 

X„ = {A;/2": A; = 0,l,...,2"-1}. 

Scaling down $* by the factor of 2^, we get a mapping g^ : Xn — >■ X^, 
for large values of n, this mapping approximates a pseudorandom number 
generator g : [0, 1) — > [0, 1). 

Long period alone does not suffice to make a pseudorandom number 
generator acceptable; in order to be acceptable, it has to pass a number 
of statistical tests. A number of these tests is commonly agreed on; our 
favourite ones are implemented in the software library TestUOl of L'Ecuyer 
and Simardfl^ [T5] . In particular, TestUOl includes batteries of statistical 
tests for sequences of uniform random numbers in the interval [0,1). The 
least stringent of them, SmallCrush, consists of the following ten tests: 
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1 smarsa_BirthdaySpacings 

2 sknuth_Collision 

3 sknuth_Gap 

4 sknuth_SimpPoker 

5 sknuth_CouponCollector 

6 sknuth_MaxDft 

7 svaria_WeightDistrib 

8 smarsa_MatrixRaiik 

9 sstring_HammingIndep 
10 swalk_RandomWalkl 

Each of these tests produces a number p; a typical range where the test is 
considered passed is 0.001 < p < 0.999. 

Is there a McCulloch-Pitts network $ whose pseudorandom number gen- 
erator passes all ten tests of SmallCrush? Our networks fail all ten. 

We conclude with an example of a 4-neuron McCulloch-Pitts network 
which has neither of the two properties (fTOj) . (fTTj) . This network is defined 
by 

(Xi, X2, ^3, X4) H-)- {H{~Xi - X2- -0:4 + 2), H{-Xi - X2+XS + 2x4 - 1), 

H{—xi + 2x2 — X3 + X4 — 1), H{2xi — X2 — x^ + X4 — 1)) 

and its trajectory is 

(0, 0, 0, 0) (1, 0, 0, 0) ^ (1, 0, 0, 1) ^ (1, 1, 0, 1) H-> 
(0, 0, 1, 1) ^ (0, 1, 0, 0) (1, 0, 1, 0) (0, 0, 0, 1) ^ 
(1, 1, 1, 1) ^ (0, 1, 1, 1) ^ (0, 1, 1, 0) (0, 0, 1, 0) ^ 
(1,1,0,0) ^ (1,0,1,1) ^ (0,1,0,1) ^ (1,1,1,0) ^ 
(0,0,0,0) (1,0,0,0) f-^ ... . 

For each subscript i = 1,2,3, the triples (xj, $(x)i, $^(x)j) run through the 
entire set {0, 1}^; the triples {x^, $(2^)4, ^^{x)^) miss only two values, which 
are (0,1,0) and (1,0,1). 
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